Flexible storage of media information

ABSTRACT

Items of media information are stored in separately reusable blocks of memory. Items are split into a plurality of layers, so that the item can be represented at reduced quality by a subset of the layers without using a remainder of the layers. Each layer is stored in a respective one of the blocks. When information of a subsequent item is stored, a block for storing the information of the second item dependent on a number of blocks that is in use for storing information. An unused block is selected when the number of blocks that is in use is below a threshold number. When the number is above the threshold number a block is re-used that was previously used for storing a layer from said remainder. Preferably, when the layers are encrypted, different layers of the first item are encrypted independently of one another, so that each layer can be decrypted even if the remainder has been overwritten.

The invention relates to a method of storing items of media information,a method of encrypting media information and a media information storagesystem.

Storage of media information, such as audio and video informationrequires a considerable amount of storage space. In storage systems suchas disk systems, semiconductor memory, tapes etc., only a finite amountof storage space is available. However, the intrinsic property of mediainformation is that it remains useful even if a certain amount ofquality is sacrificed. To increase the amount of media information thatcan be stored in a storage system, the information is preferablycompressed at the expense of quality. Many techniques have beendeveloped to achieve a maximum compression ratio at the expense of aminimum loss of quality. Nevertheless, even with optimum techniques, thequality of the image will ultimately decrease with increased compressionratio.

European patent application No. 1001632 describes a compressionapparatus that uses a variable compression ratio to manage the use ofthe available storage space. When storing incoming information, thecompression ratio is adapted so as to ensure that an entire TV programfits into the available memory space. The adaptation of the compressionrate is dynamical, less critical parts of the program being compressedmore than more critical parts of the program. Weight information helpsthe system to forecast which parts are less critical and which are morecritical, so that the compression ratio can be adapted accordingly.

A disadvantage of this system is that it requires foresight about theamount of information that is yet to come, and its required compressionquality.

Amongst others, it is an object of the invention to provide for anefficient method of storage of media information and for a system forperforming such storage in which no foresight is necessary to adapt thecompression ratio to the available amount of storage space.

The method according to the invention is set forth in claim 1. Accordingto the invention the media information is split into different layers,for example a series of layers with additional information of increasingdetail in successive layers. Different layers for the same item(generally an interval of a TV program, such as a frame, or the like)are stored in separately reusable blocks of the memory system.Subsequently, when additional items of information are received and itturns out that insufficient memory space is available, memory can bemade available by sacrificing the content of a memory block thatcontains a layer of an earlier item (e.g. a layer with additionaldetails) that only contributes to increasing the quality with which thisitem can be recovered from memory. Thus, the quality of previouslystored items can be reduced retroactively simply by sacrificing thecontent of blocks of memory.

The method is particularly suitable for a variable length compressionsystem, in which the amount of storage space needed for the mediainformation depends on the content of the media information. In such asystem it is generally not predictable in advance how much storage spacewill be needed. By adapting the number of stored layers retroactivelysuch a system can adapt to the content of the media information.

In an embodiment, the items are encrypted so that access to the itemscan be controlled, for example to prevent copyright infringement.Preferably, the layers of an item are encrypted separately, so that itremains possible to decrypt an item even when one or more layers of theitem have been discarded. Thus, it is not necessary to decrypt theentire item and to re-encrypt it with a reduced number of layers when anumber of layers is discarded.

Of course, the layers need not be encrypted for this purpose at the timeof storage. The encryption of the individual layers may be performedseparately, for example before transmission or transport of the mediainformation to the storage system.

These and other objects and advantages of the method and systemaccording to the invention will be disclosed in more detail using thefollowing figures.

FIG. 1 shows a storage and retrieval system;

FIG. 2 shows a flow-chart of a method of storing information.

FIG. 1 shows a storage and retrieval system. The system contains amemory 10, a splitter 12, a memory management unit 14, a reconstructionunit 16, encryption units 18 and a decryption unit 19. The splitter 12has an input for receiving media information, outputs for outputtinglayers of information to memory 10 via encryption units 18 andconnection with memory management unit 14. Memory management unit 14 hasan output coupled to memory 10. Reconstruction unit has an input coupledto memory 10 via decryption unit 19.

FIG. 2 shows a flow chart of the operation of the system of FIG. 1. In afirst step 21, splitter 12 receives items from a stream of mediainformation. The media information is for example video informationand/or audio information, or more generally any type of information thatcan be compressed with a gradually increasing loss of quality as morecompression is used. In the case of video information, the incomingmedia information is for example in MPEG format, encoded in terms of anumber of DCT (Digital Cosine Transform) coefficients, or in terms of aset of numbers that represent the values of individual pixels. In caseof audio, the media information is encoded for example by a number ofsample values (in PCM format) or by some compressed information, such assample values of the signal from different frequency bands. The itemsmay represent individual frames, image blocks or time intervals of themedia information, different television programs etc.

In a second step 22, splitter 12 splits the item into a number oflayers, if necessary after compression. The layers may be viewed as asuccession of layers, each adding information to the information of thepreceding layers, so that the item can be reconstructed with increasingaccuracy by using more layers of the succession.

In one example, DCT coefficients for different spatial frequencycomponents of an image are encoded in different layers, so that lowfrequency components are encoded in a first frequency components areencoded in a second layer. Thus, by using only the first layer torecover the item, one would get a low frequency (low spatial resolution)version of the image. The different coefficients may be obtained forexample by computing a DCT of an image block, or they may be taken froman MPEG signal, which encodes different DCT coefficients. In anotherembodiment, bits of different significance levels from numbers thatrepresent samples of the signal are placed in different layers. Inaddition to the split into layers, the second step 22 may also includecompression of the item, for example loss-less compression and/orvariable length compression which produces a variable amount ofinformation in each layer, dependent on the content of the item.

In another example, a first layer contains data representing samplevalues of a subsampled version of an image, a second layer containinginformation about the difference between the subsampled version of theimage and the image itself. Of course this could be repeated by placingsample values of increasingly less subsampled versions of the image insuccessive layers. In yet another example different layers containI-frames and B or P-frames of encoded MPEG images respectively.

Preferably, encryption units 18 encrypt the various layers, eachindependently from the other layers for the same item, in the sense thatthe different layers can be decrypted in the absence of other layers. Itwill be appreciated that, although different encryption units 18 areshown to emphasize the independence of encryption of the differentlayers, in practice a single encryption unit may be used to encrypt eachof the layers.

In a third step 23, splitter 12 signals to memory management unit 14 howmany blocks from memory 10 are needed to store the layers for the item.Memory management unit 14 determines whether sufficient unused blocksare available. If so, a fourth step 24 is executed, in which memorymanagement unit 14 selects a number of blocks from memory 10 for storageof the respective layers. Memory management unit 14 stores informationindicating which type of layer (e.g. the position of the layer-in thesuccession of the layers in order of increasing detail) is stored inwhich block. In a fifth step 25 splitter 12 stores the layers for theitem in memory 10. In a sixth step splitter 12 determines whether a newitem has arrived and if so the process repeats from the first step 21.

After the third step 23, if memory management unit 14 has determinedthat insufficient memory is available, memory management unit 14executes a seventh step 27, selecting a number of memory blocks forreuse. These memory blocks are already in use for layers from precedingitems. These memory blocks are selected according to the informationthat indicates which type of layer is stored in the blocks. Generally,blocks are selected that store layers which add the least amount ofquality to the items they represent. For example, using the successionof layers with additional information that adds to layers that precedethe layer in the succession, the layers at the end of the succession(that add the last additional information) are selected. The selectedmemory blocks are marked as unused, thus effectively discarding thelayers stored in those blocks. Subsequently the process continues fromthe fourth step, i.e. the selection of unused memory blocks for storageof the layers produced by the splitter 12. In an embodiment, the memorymanagement unit 14 may also signal to splitter 12 that some of the lessimportant layers of the present item may not be stored, so that lessstored layers need to be discarded.

Subsequently, or concurrently, reconstruction unit 16 reads the layersfrom the memory 10 and reconstructs the items, as far as possible, fromthe layers that have been stored and retained in the memory 10. Thequality with which reconstruction unit can reconstruct the items dependson the number of layers that have been retained. For example, lessdetail may be present in reconstructed images, as layers with finerdetails have been discarded. The reconstruction unit 16 may for examplecontain a display screen (not shown) to display the images encoded bythe items and/or a loudspeaker to output sound encoded by the items. Butthe reconstruction unit 16 may also have an output to transmit theretrieved information via a communication channel.

If necessary, decryption unit 19 is used to decrypt the layers. Becausethe layers have been encrypted independently, they can be decrypted evenif some of the less important layers have been discarded.

It will be appreciated that numerous changes can be made to theembodiment without deviating from the invention. For example, instead ofusing a separate splitter 12, memory management unit 14 andreconstruction unit 16, one or more of these functions may be combinedand/or performed using a suitably programmed general purpose processor.The memory 10 may in fact comprise a set of different memory-devices,different layers or different items being stored in different devices.Additional memory may be present, for example, the memory 10 may have acache memory function to speed up a larger main memory (not shown). Inthis case, layers are discarded to reuse their cache memory blocks, butthe main memory may retain the discarded layers. Thus, in this case, thecache replacement strategy is based on layer number of informationrepresenting items of media information rather than on parameters suchas most recent use. The memory may be implemented in any way that allowsreuse of memory blocks, for example as a semiconductor memory, forexample a DRAM or a flash memory, magnetic or optical disk, tape etc.

Various strategies may be used in the third step 23 and seventh step 27to decide when and how to discard layers. For example, in a firstembodiment, the memory management unit 14 may store a predeterminedthreshold value of freely available memory blocks in memory 10. In thisembodiment memory management unit 14 also counts how many blocks are inuse for stored information. Once the result of this count exceeds thethreshold, memory management unit 14 proceeds with the seventh step 27.But there are many alternatives to this scheme. For example, thethreshold might be set individually for storing specific televisionprograms, or more generally media files, so that a maximum amount ofstorage is allocated to each program separately. This allows a morerefined control over memory usage. The threshold might be set equal tototal memory capacity, so that reuse occurs only when the memory isfull, or the threshold might be set lower that this total, so thatalways some memory space remains available. Comparison with thethreshold need not be related to storage of new items, but it might beperformed only periodically so as to create sufficient usable memoryspace for the maximum possible amount of information to may have to bestored in a period of time.

Similarly, in the seventh step 27, selection of layers that are to bediscarded may be performed in many ways. In principle any randomselection of layers that add least to quality may be performed. However,in an embodiment weight values may be assigned to different layers, forexample according to layer position in the series of layers thatrepresents an item, importance of the item (as indicated for example bythe source of the item, or by the user of the system), expected use ofthe item etc. In this case the layers with minimum weight are discardedfirst. That is, the weight according to layer position, which is thebasis for selectively discarding layers the same item, may bedifferentiated for different items according to a weight assigned to theitem.

It will be clear that, although a illustrative embodiment of the storagesystem has been shown, the invention is not limited to that embodiment.For example, instead of using a separate splitter 12, management unit 14and encryption unit 18, one ore more of these units may be combinedand/or their function may be implemented using a suitably programmedcomputer. In that case, a program carrier such as a CD-ROM or aninternet signal may be used for example to enter the required programinto the computer. Also, although splitting and reconstruction are shownto be performed by components of the same system, it will be appreciatedthat the writing side 10, 12, 14, 18 of the system and the reading side16, 19 may be both part of a single apparatus, or part of differentapparatuses, the memory 10 being used as an intermediary in thetransport of information between the apparatuses.

1. A method of storing items of media information in a storage systemthat contains separately reusable blocks of memory (10), the methodcomprising splitting the information of a first item into a plurality oflayers, so that the first item can be represented at reduced quality bya subset of the layers without using a remainder of the layers; storingeach layer in a respective one of the blocks; storing information of asecond item subsequently to storing information of the first item;selecting a block for storing the information of the second itemdependent on a number of blocks that is in use for storing information,an unused block being selected when the number of blocks that is in useis below a threshold number, a block used for storing a layer from saidremainder being re-used for storing the information of the second itemif the number is above the threshold number.
 2. A method according toclaim 1, wherein the media information is visual information, the subsetand the remainder of the layers containing a low resolution part and aremainder of the visual information respectively.
 3. A method accordingto claim 1, comprising encrypting the items of media information,different layers of the first item being encrypted independently of oneanother.
 4. A method of encrypting media information, the methodcomprising splitting the information into a plurality of layers so thatthe media information can be represented at reduced quality by a subsetof the layers without using a remainder of the layers, the layers beingencrypted independently of one another.
 5. A media information storagesystem, comprising splitting means (12), for splitting items of receivedmedia information into a plurality of layers so that the mediainformation can be represented at reduced quality by a subset of thelayers without using a remainder of the layers; separately reusableblocks memory (10); a memory management system (14) for assigning blocksto respective layers of respective items, the memory management system(14) being arranged to select to store layers of information of a secondone of the items either in freely available ones of the blocks or inblocks in use for the remainder of the layers of a first one of theitems, said selection being dependent on a number of blocks that isfreely available, so that freely available ones of the blocks areselected when more than a threshold number of blocks is freelyavailable.
 6. A media information storage system according to claim 5,comprising encryption circuitry (18) arranged to encrypt different onesof the layers for the first one of the items independently of oneanother.
 7. A computer program product containing instructions forcausing a computer to execute the method according to claim 1.